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PREFACE 


This document reports on the progress attained during the first six 
months of activity under Contract No. NAS5-21814 from September, 1972 
through February, 1973. The purpose of the contract is to demonstrate 
precision geometric and radiometric correction of bulk digital ERTS 
imagery by digital methods, and to evaluate the techniques utilized 
from the point of view of precision and speed. Included in this report 
are brief descriptions of the processing methods applied to ERTS imagery, 
examples of imagery so far processed and a description of the program 
for the next reporting interval (March-April , 1973). To date, the 
principal items of software required to correct bulk digital image tapes 
have either been developed, or adapted from previously extant software 
for the ERTS requirements. MSS imagery have been processed by means 

of this software with good results. A Data Analysis Plan will 
shortly be submitted to NASA describing the analyses to be undertaken 

during the final phase (Phase III) of this contract. 
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1.0 INTRODUCTION 


This document summarizes key results attained during the first six 
months under contract. Also presented is an outline of the processing 
techniques employed including, in particular, the resampling technique 
for geometric and photometric correction of imagery. Examples of MSS 
imagery processed by these methods are illustrated here, and were con- 
tained in a paper presented at the Symposium on Significant Results 
Obtained from ERTS-1 , March 5-9, 1973. Processing times have been 
demonstrated to be comparable to tape read/write times. Error analyses 
of the performance of a Kalman filter used with Ground Control Points 
(GCP's) have also been performed. 

Section 2 reviews the achievements of the first four months of the 
contract, and which were reported on earlier. This section also con- 
tains a brief description of the processing approach taken by TRW. 
Section 3 reports on progress during the past two months, and contains 
examples of the first results. Section 4 describes the activities 
planned for the next two months, and Section 5 summarizes conclusions 
of the study to date. 
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2.0 REVIEW OF ACTIVITY: 
September-December, 1972 


2.1 Approach 

In order to understand the significance of the results described 

below, it would be useful to describe the approach TRW has taken to 

the problem of digitally correcting ERTS images. A schematic diagram 

* 

of the MSS software is given in Figure 2-1. It may be seen that two 
processing passes of the image data are required in this approach: 

(1) first the bulk data tapes are reformatted to produce a single band/ 
scene on individual files of data on a Computer Compatible Tape (CCT); 

(2) the reformatted image data is processed by means of a "resampling" 
algorithm, which combines both geometric correction (rectification) 
and radiometric (intensity) correction in a single step. The neces- 
sary parameters for this resampling operation are derived during the 
first pass processing (during which the bulk tapes are reformatted) 
utilizing annotation tape data (attitude, ephemeris) alone, or in 
combination with GCP attitude refinement. 

The attitude refinement is accomplished by means of a Kalman fil- 
ter which accepts GCP location data and attitude/ephemeris data as in- 
puts and produces refined attitude outputs. Either the refined attitude 
resulting from GCP processing or the initial attitude data alone are 
input to a distortion coefficient calculation code, which generates the 
coefficients which are input to the second pass program. The distor- 
tion coefficients define the amounts by which the bulk image must be 
"distorted" so as to produce a rectified image, i.e., one which is linear 
in meter/meter coordinates. 


★ 

The RBV software is functionally very similar, and thus it will 
not be necessary to discuss it any further at this time. The principal 
difference is the additional requirement for reseau identification, 
which is accomplished by means of a shadow casting technique. . 
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Figure 2-1. Precision Correction Software Schematic 
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The computation of the second pass distortion coefficients is ac- 
complished by mapping a small number of image locations (lines/pixels) 
called "pseudo reseaux" to the desired meter/meter coordinate system. 

To do this it is necessary to include the effects of mirror scan, space- 
craft attitude, velocity and altitude, sensor pointing error, earth 
rotation and earth curvature. Utilizing the mapping between input 
(bulk) and output (final) image coordinate systems, distortion coef- 
ficients are defined for each block of four pseudo reseaux so as to 
map the input image to the desired rectified coordinate system. A piecewise 
bilinear spatial distortion model is used, of the form 


6 X = a Q + a-|X + a 2 y + a 3 xy 
6^ = bg + b-jX + b 2 y + b 3 xy 



and y directions, respectively. It has been shown at TRW that for a 
sufficient density of pseudo reseaux, the errors associated with using 
a bilinear distortion model can be made arbitrarily small. In particu- 
lar, a reasonable number of pseudo reseaux (less than 100 for an en- 
tire image) will produce model errors less than the self-consistency 
errors (all other errors associated with the mirror scan, ephemeris 
and attitude, etc. ) . 

All of the calculations just outlined are straightforward and can 
be accomplished very quickly (^ 5 sec CPU for an entire image), with 
the possible exception of the GCP identification. One standard method 
is to cross correlate chosen subareas of the bulk image against a refer- 
ence template image. Typical times for a subarea 120 pixels x 120 
lines and a template ^ 30 pixels x 30 lines are ^ 10 sec. TRW will 
examine and evaluate alternative algorithms in the course of the re- 
mainder of this contract which offer significant improvements in run- 
ning time. 

The more time consuming part of the processing is the second 
pass image "resampling." Conceptually, this operation consists of 


4 



reconstructing in its entirety the corrected (rectified) image and 
then sampling its intensity values at predesignated positions (lines/ 
pixels) within the image. It should be clearly understood that at no 
time is the input image data physically distorted, with data values subse- 
quently defined by interpolation. The power of the resampling method becomes 
clear when it is realized that complete flexibility is afforded in 
the matter of pixel spacings and line spacings (independently), thus 
making possible the custom tailoring of the corrected image to any ap- 
plication and/or hardware constraints. 

The resampling method is implemented by means of a convolution 
algorithm. Ideal one-dimensional resampling of band-limited digital 
data is of the form 


I(x.) = L Kx k ) f ( x r x k^ 
x k 

wherein f(x) is of the form sinx/x, x. is the argument (pixel location) 
chosen for the resampled function I, and the set of x^ is the set of 
arguments of the available digital data I(x^), that is, the bulk data 
values. TRW has approximated this ideal sinx/x resampling by means of 
a continuous symmetric cubic polynomial function, possessing a continu- 
ous first derivative and vanishing beyond + 2 pixels (lines). Subject 
to these and a few other constraints, a unique function results, as 
shown in Figure 2-2. For obvious reasons it is called the cubic 
convolution function." It was shown in the TRW proposal leading to 
the current contract that cubic convolution results in very small er- 
rors compared to other methods for generating I(x,.). These other 
methods for obtaining the value of a function at a point given values 
of the function at nearby points is called interpolation, and is il- 
lustrated in Figure 2-3 for nearest neighbor and bilinear interpolation. 

Nearest neighbor interpolation involves choosing the value of the 
pixel closest to the desired pixel location as the interpolated value, 
as shown in Figure 2-3-a. Only one pixel value of the bulk image is 
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Figure 2-2. Cubic Convolution Interpolation Function 



(a) Nearest Neighbor Interpolation 



(b) Bilinear Interpolation 


Figure 2-3. Two Possible Interpolation Algorithms. Circles 
denote uniform pixel locations in the output, corrected image, 
x's denote the irregular grid of pixels (in the corrected 
image), derived by straightforward corrective distortions to 
the input bulk image. Fill ed -i n zero is a typical pixel, 
the value of which is to be interpolated. 
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required for each interpolated value, and only original values are 
used. The algorithm is thus very fast. Note however, that errors 
accumulate in finite (+1/2 pixel) increments and thus will result in 
1 pixel offsets throughout the corrected image. 

Bilinear interpolation, shown in Figure 2-3-b, utilizes a bilinear 
combination of the four closest pixel values to produce a new, inter- 
polated pixel value. The smoothing effects of bilinear interpolation 
preclude the one pixel offsets characteristic of nearest neighbor 
interpolation. By the same token, this smoothing effect can cause a 
small amount of image degradation. Also, the algorithm is inherently 
slower, inasmuch as a new pixel value must be computed from four other 
values. 

In contrast to nearest neighbor and bilinear interpolation, cubic 
convolution resampling requires the values in a grid 4 pixels x 4 lines 
about the point at which the interpolated value is desired. Thus, both 
slope and continuity properties of the function to be resampled are pre 
served (and expected on the basis of its approximation to ideal sinx/x 
resampling). As a consequence, high resolution with minimum distortion 
is anticipated, with some penalty in CPU time compared to bilinear in- 
terpolation. It will be seen in Section 3, however, that quite ac- 
ceptable CPU times have been attained with TRW's implementation of 
cubic convolution resampling. 

2.2 Achievements 

During the first two months of the contract, September-October, 
1972, simulated data was used to exercise components of the RBV soft- 
ware and MSS software. No imagery tapes were received during the 
period, and the first system corrected images from our selected test 
sites became available the last week of the period. 

During November-December, 1972, the first ERTS image tapes were 
received. GCP extraction from an ERTS scene was tested, and a Kalman 
filter for precision attitude determination was adapted for the ERTS 
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application and tested against simulated data. An annotation tape be- 
came available during this period and was successfully utilized with 
the software for the computation of spacecraft attitude and ephemeris. 
The second pass resampling program was adapted to ERTS requirements, 
and tests using it with ERTS data were initiated. 

Some appreciation for the performance of the Kalman filter in 

refining attitude may be gained with reference to Figures 2-4 and 

2-5. The first figure shows the standard deviation in equivalent 

degrees for terms in the pitch state vector, as a function of the 

• • • *• • 

number of GCP's processed. The pitch state vector (Pg, Pg, P Q ) cor- 
responds to a cubic polynomial of the form: 

. P 0 n Pft o 

P = Pg + Pgt + 2 J t + 3 T t (degrees) , 

which is fit to annotation tape data. The standard deviations of the 
pitch rate, pitch acceleration and pitch rate of acceleration have been 
normalized to the same value to facilitate comparisons. Figure 2-5 
shows a corresponding plot for the yaw state vector. 

It may be seen that the zero-order pitch term is better estimated 
than the zero-order yaw term. This is due to the greater sensitivity 
of the GCP error terms (input to the filter) to pitch than to yaw devia 
tions and can be demonstrated in a straightforward manner from geometri 
cal considerations. 

The results presented here were generated from simulated data, 
using a particular pattern of GCP locations. Filter performance is 
somewhat dependent upon GCP locations; better performance is obtained 
when the points are not clustered. Also, important inputs to the fil- 
ter are the covariance matrix for the pitch, roll and yaw state vec- 
tors. In lieu of any measurement data, TRW has estimated these values 
from attitude data supplied by NASA. 
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Figure 2-5. Standard Deviation in Equivalent Degrees for the YAW State 
Vector (Y q/ Y q/ Y q/ Y q ) Estimated at Five GCP Updates. 
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2.3 Problems 

During the first reporting interval the chief problem was lack 
of data and imagery. Lack of annotation tape (BIAT or DIAT) data 
constituted the chief difficulty during the second reporting period. 
Also, the Monterey Bay scene was added to our 
previous list of sites, with attendent delays in obtaining imagery 
and tapes. Mirror scan calibration data and the MSS line length 
adjustment algorithm similarly were not available during this inter- 
val. The lack of such data does not significantly impede progress, 
but will impact the quality of the final images produced. 
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3.0 ACTIVITY: 
January-February, 1973 


3.1 Achievements 

By the middle of this reporting period, annotation data had been 
received for a scene containing San Francisco Bay and Monterey Bay and 
for which system corrected imagery and bulk digital data tapes had 
been received earlier. The first image corrected by TRW was scene 
1057-18172-7. This infrared band was chosen specifically to bring 
out clearly the comparative effects of nearest neighbor interpolation, 
bilinear interpolation and cubic convolution resampling in high con- 
trast regions of the image, i.e., land/water interfaces. 

Figure 3-1 is a reproduction of the system corrected image for 
the entire scene supplied by NASA/GSFC. Note that this image has been 
corrected for the unequal number of lines (2340) and samples/line 
3240) in the bulk data, as well as earth rotation (which produces 

the ^3° skew). The image has also been contrast enhanced by NASA. 

. * 

Figure 3-2 shows a detail taken from the upper left corner of 
the same image, reconstructed** from the unrectified bulk image data 
tape supplied by NASA. The area shown is approximately 45 Km across 
and 50 Km down. 

A rectified image corresponding to the detail in Figure 3-2 is 
given in Figure 3-3, utilizing the nearest neighbor interpolation al- 
gorithm and without precision GCP attitude refinement. The area shown 
is 46.25 Km x 36.4 Km, and is aligned parallel to the spacecraft ground 
track. Note the steps occurring on the left edge of the image, cor- 


* 


The pictures included here have been slightly contrast enhanced 
as to bring out the image detail more clearly. The photometrically 
corrected data is, of course, preserved on the digital output tape. 
** 


so 


The reconstructed images were produced from the corrected image taps 
by photographing directly a high resolution CRT face, at the Jet Propul 
sion Laboratory s Image Processing Laboratory. The bright area in the 
upper left corner is a result of the reconstruction process, and is not 
in the corrected image data. 


13 



zoom a) • i 



Figure 3-1. Reproduction of Scene 1057-18172-7 Bulk Image 
Print. The image has been corrected for the difference in 

the number of pixels/line and the number of lines, and has 

also been corrected for earth rotation (skew). 
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Figure 3-2. Reconstruction of Bulk Image Tape. This detail 
is taken from the upper left corner of Figure 3-1, and shows 
San Pablo Bay in the middle. 





responding to the successive one pixel increments associated with near 
est neighbor interpolation. Careful examination of the image reveals 
a number of areas for which this pixel jitter is observed, and which 
is particularly evident for such high contrast regions as those con- 
taining a boundary between land and water. Thus, for example, note 
the bends in the San Antonio Creek, leading off from the WNW corner 
of San Pablo Bay, located in the upper middle of Figure 3-3, 


Figure 3-4 shows the same scene as Figure 3-3, but using bi- 
linear interpolation instead of nearest neighbor interpolation. Note 
the straight line appearance of the left edge of the image, in contrast 
to Figure 3-3. Note also that the bends in the San Antonio Creek are 
cleaner. On the other hand, note the slight resolution degradation 
for the complex of roads (including Interstate 80) north of Carquinez 
Strait, and east of Mare Island and the Napa River, compared with 
Figure 3-3. This degradation is expected by virtue of the smoothing 
effect of bilinear interpolation, and is absent in nearest neighbor 
interpolation. 

* 

Figure 3-5 shows a portion of the same area resulting from the 
cubic convolution resampling algorithm . Detailed comparisons between 
Figures 3-4 and 3-5 show a number of areas in which the resolution re- 
sulting from cubic convolution is superior to bilinear interpolation, 
while at the same time there is no evidence of the pixel offsets as- 
sociated with nearest neighbor interpolation. 


* 

In order to obtain this picture, a non-standard development process 
was utilized, compared to Figures 3-2 through 3-4. The resulting con- 
trast was very poor, and could not be entirely compensated for in this 
reproduction. 

The area shown is one-half the width of the previous examples due 
to a peculiarity in the processing method completely extraneous to 
the resampling algorithm. The corrected digital data tape for this 
image does not possess any such defects. 
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Figure 3-5. Reconstruction of a Portion of Figure 3-2 After 
Rectification, Using Cubic Convolution Resampling. 


It is worthwhile emphasizing here that the rectification proces- 
sing was accomplished by all digital processing methods, utilizing in- 
put digital imagery. In contrast to this, bulk imagery such as in Figure 
3-1 is produced at the NASA Data Processing Facility by converting digital 
data to analogue data, and then utilizing an Electron Bean Recorder to re- 
scan the image onto the output film. 

In addition, it is useful to point out the required processing 
times for the production of rectified imagery. The images shown in 
Figures 3-3 through 3-5 were processed on a CDC 6400. Exclusive of 
the resampling portion, the required processing time was less than 
10 seconds. Including GCP processing, the time required is of the 
order of a minute or two (depending on the number of GCP's). The 
more time consuming portion is the resampling algorithm, which requires 
approximately 52 seconds of CPU time per 10 6 pixels for bilinear inter- 
polation. Cubic convolution requires approximately 110 seconds. In 
contrast, extrapolations to a PDP-1 1/45 minicomputer indicate 28 sec- 
onds of CPU (per 10 6 pixels) would be required for bilinear interpola- 
tion, or approximately 60 seconds of CPU for cubic convolution. Thus, 
for an output image consisting of 10 7 pixels, 10 minutes of processing 
time would be required for cubic convolution on the minicomputer (or 
less than 5 minutes for bilinear interpolation), a time comparable to 
that required to read a CCT or write a CCT. 

3.2 ERTS-1 Symposium Participation 

During the March 5-9 Symposium on Significant Results Obtained 
from ERTS-1, TRW presented a paper containing some of the results 
described above. 

3.3 Problems 

Difficulties continue to be experienced in receiving the annota- 
tion data, corresponding bulk digital tape£ and system corrected images 
(for initial processing runs), in a timely manner. In addition, the 
current lack of an in-house image hardcopy capability is producing 
significant delays in some of the evaluations of image quality. 
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4.0 PROGRAM FOR MARCH-APRIL, 1973 


During the next reporting interval, it is planned to generate a 
Data Analysis Plan describing activities to be performed during the 
final (Phase III) portion of this contract. GCP refinement of the 
spacecraft attitude will be used for precision processing. Alterna- 
tive algorithms for GCP identification will be exercised and evaluated 
from the point of view of speed and precision. During Phase III of 
the contract, RBV imagery will also be processed and statistical as 
well as other numerical methods will be applied to evaluate the geo- 
metric and radiometric precision of the final image products. 
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5.0 CONCLUSIONS 


Early results have been presented for the digital rectification 
of ERTS imagery. Two interpolation algorithms were employed to pro- 
duce the final corrected imagery included here. It was shown that 
small errors 1 pixel, or < 60 m) result from the resampling pro- 
cess, for only modest CPU requirements. Nearest neighbor interpolation 
produces +1/2 pixel jitter (horizontal and vertical) which is evident 
in regions of high contrast. On the other hand, bilinear interpolation 
supplants the single pixel offsets with smooth transitions, but at 
the price of somewhat larger CPU times and some slight resolution de- 
gradation. The cubic convolution resampling algorithm combines the 
best properties of nearest neighbor and bilinear interpolation, with 
only modest additional CPU requirements over those for bilinear re- 
sampling. 
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